Lens-attached matter detector, lens-attached matter detection method, and vehicle system

ABSTRACT

A lens-attached matter detector includes an edge extractor configured to create an edge image based on an input image, divide the edge image into a plurality of areas including a plurality of pixels, and extract an area whose edge intensity is a threshold range as an attention area, a brightness distribution extractor configured to obtain a brightness value of the attention area and a brightness value of a circumference area, a brightness change extractor configured to obtain the brightness value of the attention area and the brightness value of the circumference area for a predetermined time interval, and obtain a time series variation in the brightness value of the attention area based on the brightness value of the attention area, and an attached matter determiner configured to determine the presence or absence of attached matter based on the time series variation in the brightness value of the attention area.

PRIORITY CLAIM

The present application is based on and claims priority from JapanesePatent Application No. 2012-149537, filed on Jul. 3, 2012, and JapanesePatent Application No. 2013-133576, filed on Jun. 26, 2013, thedisclosures of which are hereby incorporated by reference in theirentirety.

BACKGROUND

1. Field of the Invention

The present invention relates to a lens-attached matter detector whichdetects attached matter on a lens of a camera, for example, alens-attached matter detection method, and a vehicle system includingthe lens-attached matter detector.

2. Description of the Related Art

A system is known which detects a vehicle or a pedestrian existing in ablind area behind a vehicle by an in-vehicle camera when changing alane, so as to draw a driver's attention by an alarm or an indicationlight. A system is also known which detects a white line on a road by acamera, so as to draw a driver's attention by an alarm or an indicationlight when a driver drifts from a lane. Further, a system which sounds ared alert by detecting a vehicle getting closer from behind and a systemwhich assists parking by detecting a parking frame are also known.Hereinafter, an in-vehicle system using an image recognition techniqueis referred to as an image-sensing application.

A camera for use in such a system may be provided outside a vehicle, andis used while the vehicle is running. For this reason, a stain such assplash of dirt attaches onto a lens surface. When such a stain isprominent, the detection accuracy of a subject by a camera isdeteriorated. For this reason, the performance of an image-sensingapplication such as a system which detects a vehicle or a pedestrian, ora system which detects a white line may be affected. Thus, when a driveris aware of a stain, such a stain is eliminated from a lens by sprayingair or cleaning liquid.

However, a driver is sometimes not aware of a stain. For this reason, astain detector, which automatically detects a stain on a lens, so as toencourage cleaning by informing a driver of the generation of the stainor automatically clean the stain, has been developed (refer to, forexample, JP 2003-259358A). In the stain detector described in JP2003-259358A, a concentration value is obtained for each of two imagesshot by a camera in different timings, and a difference of these valuesis extracted to be integrated, so as to obtain an integrated image. Insuch an integrated image, an area having a predetermined concentrationvalue or below, namely, an area without having a change over time isdetermined as an area to which a stain is attached. When thisstain-attached area is an area for use in an image process by a camera,it is determined that a lens is stained, and a stain-attached signal issent to a driver or a cleaner.

However, in the stain detector described in JP 2003-259358A, a landscapeor an object to be shot (for example, long guide rail, parapet, or sidewalk) having less change over time has an increased concentration valueof an integrated image with a difference, causing false-determination asa stain. Therefore, the development of a highly accurate technique whichcan detect only attached matter on a lens such as a stain is requested.

SUMMARY

The present invention has been made in view of the above circumferences,and an object of the present invention is to provide a lens-attachedmatter detector, lens-attached matter detection method capable ofdetecting only lens-attached matter such as dirt, dust or waterdropswith a high accuracy, and a vehicle system having the lens-attachedmatter detector.

To attain the above object, one embodiment of the present inventionprovides a lens-attached matter detector, including: an edge extractorconfigured to create an edge image based on an input image from animager having a lens, divide the edge image into a plurality of areasincluding a plurality of pixels, and extract an area whose edgeintensity is a threshold range as an attention area; a brightnessdistribution extractor configured to obtain a brightness value of theattention area and a brightness value of a circumference area; abrightness change extractor configured to obtain the brightness value ofthe attention area and the brightness value of the circumference areaobtained by the brightness distribution extractor for a predeterminedtime interval, and obtain a time series variation in the brightnessvalue of the attention area based on the brightness value of theattention area for the predetermined time interval; and an attachedmatter determiner configured to determine the presence or absence ofattached matter based on the time series variation in the brightnessvalue of the attention area.

Moreover, one embodiment of the present invention provides alens-attached matter detection method which is executed by the abovelens-attached matter detector, including an edge extraction process ofcreating an edge image based on an input image, dividing the edge imageinto a plurality of areas including a plurality of pixels, andextracting an area whose edge intensity is a threshold range as anattention area; a brightness value distribution extraction process ofobtaining a brightness value of the attention area and a brightnessvalue of a circumference area of the attention area; a brightness valueextraction process of obtaining the brightness value of the attentionarea and the brightness value of the circumference area obtained by thebrightness value distribution extraction process for a predeterminedtime interval, and obtaining a time series variation in the brightnessvalue of the attention area based on the brightness value of theattention area for the predetermined time interval; and an attachedmatter determination process of determining the presence or absence ofattached matter based on the time series variation in the brightnessvalue of the attention area.

Furthermore, one embodiment of the present invention provides a vehiclesystem, including the above lens-attached matter detector; an imagerprovided in a vehicle, having a lens, and configured to image acircumference of the vehicle; and at least one application configured tooperate based on detection information of the lens-attached matterdetected by the lens-attached matter detector relative to the inputimage shot by the imager.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings are included to provide further understandingof the invention, and are incorporated in and constitute a part of thisspecification. The drawings illustrate embodiments of the invention and,together with the specification, serve to explain the principle of theinvention.

FIG. 1 is a schematic view illustrating a vehicle equipped with avehicle system including a lens-attached matter detector according tofirst and second embodiments of the present invention.

FIG. 2 is a block diagram illustrating the configuration of the vehiclesystem including the lens-attached matter detector according to thefirst and second embodiments of the present invention.

FIGS. 3A, 3B are views describing a step of blocking a process targetimage, FIG. 3A illustrates areas for use in vehicle detection and lanedetection in an input image, and FIG. 3B illustrates blocked areasincluding these areas.

FIG. 4 is a view describing a relationship between an input image (a)input to an edge extractor and an edge image (b) output from the edgeextractor.

FIG. 5 is a view describing a step of obtaining a brightness average ofprocess blocks.

FIGS. 6A, 6B are views describing a step of counting a brightcircumference block, FIG. 6A illustrates an attention block and itscircumference blocks, and FIG. 6B illustrates circumference blocks whichare targets for the counting step in the second embodiment.

FIG. 7 is a view describing a step of counting the number of weak edges,and also is a schematic view illustrating a block having a weak edge.

FIG. 8 is a view describing accumulation of brightness value informationin chronological order, and illustrating an image of record which isrecorded in a memory.

FIGS. 9A, 9B, 9C are views describing a mask process for a lane, FIG. 9Aillustrates a schematic view of an edge filter for a left area, FIG. 9Billustrates a schematic view of an edge filter for a right area, andFIG. 9C illustrates an image of a lane by real view.

FIG. 10 is a view describing a mask process for a lane, and illustratinga step of generating an edge image by applying a filter in each of theright and left areas.

FIG. 11 is a view describing a mask process for a track, andillustrating an input image (a) having a track, an image (b) in whichthe input image is divided into a plurality of frames, and the edges aredetected, and a track mask image (c) generated by a mask processor, andan output image.

FIGS. 12A, 12B are views describing a mask process for a light source,FIG. 12A illustrates an input image having a west sun, and FIG. 12Billustrates a light source mask image generated by the mask processor.

FIG. 13 is a view describing a mask process for an own vehicle shadow,and illustrating an input image (a) having an own vehicle shadow and anown vehicle shadow mask image (b) generated by the mask processor.

FIG. 14 is a view describing a mask process for a vehicle body, andillustrating an input image (a) having a vehicle body and a vehicle bodymask image (b) generated by the mask processor.

FIG. 15 is a schematic view illustrating a weak edge image before themask process and a weak edge image after the mask process.

FIG. 16 is as flowchart illustrating flow of a lens-attached matterdetection process in the first embodiment.

FIG. 17 is a flowchart illustrating flow of a process area-obtainingprocess.

FIG. 18 is a flowchart illustrating flow of an edge extraction process.

FIG. 19 is a flowchart illustrating brightness distribution extractionprocess.

FIG. 20 is a flowchart illustrating flow of an attached matterdetermination process.

FIG. 21 is a flowchart illustrating flow of a lens-attached matterdetection process in the second embodiment.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Hereinafter, an embodiment of a vehicle system including a lens-attachedmatter detector according to the present invention will be describedwith reference to the drawings.

First Embodiment

FIG. 1 is a view illustrating a running vehicle 1 equipped with avehicle system including a lens-attached matter detector according tothe first embodiment. A camera 20 is provided in the back of the vehicle1. The camera 20 has a shooting angle of view to which can shoot backover a wide range, so that the camera 20 can shoot a blind area of adriver. A vehicle detection application or the like is performed byusing the camera 20. When the vehicle 1 runs in a traffic lane L₂ of athree-lane road 2 having lanes L₁-L₃, and changes a traffic lane fromthe traffic lane L₂ to the traffic lane L₁, another vehicle 3 in theblind area in the back of the vehicle 1 is detected. The vehicledetection application which detects another vehicle 3 starts to beep orcalls attention to a driver by lighting an indication light.

In this embodiment, the lens-attached matter detector is used for thecamera 20 provided in the back of the vehicle 1, but the presentinvention is not limited thereto. When a camera is provided in the frontor the side of a vehicle, any lens-attached matter of these cameras canbe detected. The lens-attached matter detector can be configured toinform which lens has attached matter.

A vehicle system 10 according to the first embodiment will be describedwith reference to FIG. 2. The vehicle system 10 according to the firstembodiment is installed in the vehicle 1, and includes the camera(imager) 20 which images an image at the back of the own vehicle 1, alens-attached matter detector 30 which detects lens-attached matter, acleaning function (cleaner) 51 which automatically cleans based on theresult detected by the lens-attached matter detector 30, an alarmgenerator 53 which informs a driver of lens-attached matter by alarm, adisplay 54 which informs a driver a lens-attached matter by a lamp orcharacter display, an image-sensing application 52 such as vehicledetection or lane detection, and a memory 60 in which a program or datafor use in each process is stored.

The camera 20 includes a lens 21, an imaging element 22 which convertsan image imaged by the lens 21 into analogue electric signals, and again adjuster 23 which adjusts a gain of the image obtained by theimaging element 22. The memory 60 can be a hard disk or an externalmemory which stores programs or data, or can be a memory whichtemporarily stores data such as RAM or ROM.

The lens-attached matter detector 30 includes an image processor 31which accumulates image information by executing various processes tothe image input from the camera 20, an attached matter determiner 32which determines the presence or absence of lens-attached matter basedon the information from the image processor 31, a vehicle informationobtainer 33 which obtains vehicle information such as a vehicle speed,an output information generator 34 which generates output information toanother processor based the presence or absence of lens-attached matter,and a memory 60 which stores the image information obtained by the imageprocessor 31, the determination result by the attached matter determiner32 or the like. In the lens-attached matter detector 30 of the firstembodiment, the memory 60 provided in the vehicle system 10 is also usedas a memory. The lens-attached matter detector 30 or the image-sensingapplication 52 can be a program which is executed by a computer having aCPU, memory, I/O, timer or the like. In the lens-attached matterdetector 30, the lens-attached matter detection process is programmed,and a repetition process is executed at a predetermined cycle.

As illustrated by the solid line in FIG. 2, the image processor 31 ofthe first embodiment includes a process area obtainer 35 which obtainsthe input image from the camera 20, and sets a process area to dividethe process area into a plurality of blocks, an edge extractor 36 whichextracts an area having a weak edge intensity from the input image, abrightness distribution extractor 37 which extracts a brightnessdistribution by obtaining a brightness value of the area having a weakedge intensity and the area therearound, and a brightness changeextractor 38 which obtains a time series variation in a brightness valuebased on the brightness value accumulated through time.

A lens-attached matter detection process in the lens-attached matterdetector 30 according to the first embodiment will be described withreference to FIGS. 3-8 describing processes and the flowcharts of FIGS.16-20. In this case, processing steps for detecting dirt aslens-attached matter will be described.

As illustrated in the flowchart in FIG. 16, the lens-attached matterdetection process of the first embodiment includes a vehicleinformation-obtaining process (step S10), vehicle speed determinationprocess (step S20), process area-obtaining process (step S30), edgeextraction process (step S40), brightness distribution extractionprocess (step S50), process time determination process (step S60),brightness change extraction process (step S70), attached matterdetermination process (step S80), and output information generationprocess (step S90). These processes are controlled by a controller(CPU).

(Vehicle Information-Obtaining Process and Vehicle Speed DeterminationProcess)

The vehicle information-obtaining process (Step S10) and the vehiclespeed determination process (Step S20) are executed by the vehicleinformation obtainer 33. The vehicle speed obtainer 33 obtains speedinformation sent from a speed sensor or the like (Step S10). Next, it isdetermined whether or not the vehicle speed is a predetermined value ormore (for example, 1 km/h or more) (Step S20). When it is determinedthat the vehicle speed is a predetermined value or more (YES), thesubroutine proceeds to the next process area-obtaining process (StepS30). When it is determined that the vehicle speed is less than 10 km/h(NO), the subroutine proceeds to the output information generationprocess (Step S90) after skipping the processes in Steps S30-S80. Afterthat, the whole lens-attached matter detection process is completed.More particularly, the attachment of the dirt easily occurs while thevehicle 2 is running. For this reason, the attachment of the dirt hardlyoccurs while the vehicle 2 is not running or is slowly running, in whichcase it is determined that it is not necessary to execute thelens-attached matter detection process.

In the first embodiment, although the criteria for determining thecontinuation or completion of the lens-attached matter detection processis one predetermined value as described above, the present invention isnot limited thereto. For example, the start and the completion can bedetermined based on different predetermined speeds, or the enginestartup or the shift change can be used as the determination criteria.Moreover, the continuation or the completion of the lens-attached matterdetection process can be determined based on vehicle information of aweather condition (for example, wet day) or a road condition (off-roadrunning) which easily occur attachment of dirt.

(Process Area-Obtaining Process)

The process area-obtaining process (Step S30) is executed by the processarea obtainer 35. The details of the process area-obtaining process willbe described with reference to the flowchart in FIG. 17. The processarea obtainer 35 firstly obtains the image input from the camera 20, andreduces the image (Step S31). However, the present invention is notlimited thereto, and the image can be used without being reduced.However, the process speed as well as the memory capacity for imageinformation or the like can be reduced by using such a reduced image inthe subsequent processes.

Next, the process area obtainer 35 sets a process target area from areduced monochrome image (Step S32). The process target area can be theentire input image, but an area including the process area of theimage-sensing application 52 for use in this embodiment (for example, aprocess area B in vehicle detection, process area L in traffic lanedetection and area which determines the execution of the automaticcleaning in the cleaner 51) can be the process target area 102 in theinput image 100 in this embodiment as illustrated in FIGS. 3A, 3B. Theaccuracy of the image-sensing application as well as the processefficiency of the lens-attached matter detection process can be improvedby detecting the lens attachment in the position including at least theprocess area of various functions or the image-sensing application 52.

Then, this process target area is divided into a plurality of blocks 201(Step S33) as illustrated in FIG. 3B. The subsequent processes areperformed in each block, so that the subsequent processes can beeffectively executed compared to a case which executes the subsequentprocesses in each pixel. In this embodiment, the size of each block 201is set in a size of desired dirt to be detected or below. By thissetting, only dirt can be securely and effectively detected. The blockinformation of each divided block 201 such as coordinates is stored inthe memory 60 in accordance with a block number.

(Edge Extraction Process)

The edge extraction process (Step S40) is executed by the edge extractor36. The details of the edge extraction process will be described withreference to the flowchart in FIG. 18. As illustrated in FIG. 18, theedge extractor 36 reduces the input image reduced by the processarea-obtaining process (Step S30), and obtains a black and whitegray-scaled image (Step S41). The effective process can be executed byreducing the image as just described. Next, the edge extractor 36executes the edge extraction of the reduced input image (Step S42). Thisedge extraction can be executed by using a known technique. A thresholdprocess is executed to edge intensity by using the extracted edge imageto extract only an edge required for this process, and then abinarization process is executed (Step S43). In the threshold process,an edge image having only an edge (weak edge) whose intensity ρ iswithin a predetermined range, for example, is generated. FIG. 4illustrates a view illustrating an edge image 103 generated from theinput image 100 by the edge extraction process. Dirt portions areextracted from the input image (a) in FIG. 4 as weak edges asillustrated in the edge image (b) in FIG. 4

In addition, the dirt when running an off road differs from the dirtwhen running an on road in a concentration or a tone, or in a degree ofweak edge intensity. The edge intensity may differ depending on a typeof an attached material. For this reason, a plurality of thresholds isprepared according to such road conditions, other running conditions,types of attached materials, attachment conditions or the like, and itmay be determined which threshold is used when executing thelens-attached matter detection.

Next, a noise elimination process (Step S44) which eliminates noisepresenting in the generated weak edge image is executed. In thisembodiment, the following edges are defined as noise.

(a) An edge in a position different from an edge position of apreviously extracted edge image.

(b) An edge whose area is a predetermined value or below.

At first, the noise of the above (a) is eliminated by obtaining AND ofthe edge image extracted in Step S43 and the previously extracted edgeimage. This is because the edge which is desired to be extracted by thelens-attached matter detection process of the present embodiment is anedge of dirt attached to a lens, and such dirt attached to the lensexists in the same position for a predetermined time, so that amomentarily extracted edge may be noise. In addition, the previouslyextracted image indicates an edge image obtained by the previouslyexecuted edge extraction process. This process is to compare the edgeimage extracted by the current edge extraction process with the edgeimage extracted by the previous process because this process is repeatedmultiple times within a predetermined time. However, in the firstprocess, there is no previous edge image, thus, the elimination of thenoise (a) can be skipped.

Next, the above edge (b) whose area is a predetermined value or below iseliminated as noise. The edge of the dirt attached to the lens isassumed to be extracted as a block, so that such an independent smalledge is determined as not being dirt. By executing the above noiseelimination, the lens-attached matter can be detected with highaccuracy.

(Brightness Distribution Extraction Process)

The brightness distribution extraction process (Step S50) is executed bythe brightness distribution extractor 37. The details of the brightnessdistribution extraction process will be described with reference to theflowchart in FIG. 19 and FIGS. 5, 6A, 6B. At first, in the brightnessdistribution extractor 37, as illustrated in FIG. 5, an averagebrightness value I_(ave) of a brightness value I of pixels in each blockset and divided by the setting process for the process target area iscalculated by the following expression (1) (Step S51). In the followingexpression (1), u, v denote x, y coordinates in a block, N, M denote thenumber of pixels in the x direction (horizontal) and in the y direction(vertical) in a block, n, m denote a position (relative coordinates) ofthe x direction (horizontal) and y direction (vertical) of a pixel in ablock, and n_(min), m_(min) denote the coordinates of the first pixel ina block. In addition, the brightness value uses a gray-scaled imagebefore binarization.

$\begin{matrix}\lbrack {{Expression}\mspace{14mu} 1} \rbrack & \; \\{{{Iave}( {u,v} )} = {\frac{1}{N \times M}{\sum\limits_{n = 0}^{N - 1}{\sum\limits_{m = 0}^{M - 1}{I( {{n_{\min} + n},{m_{\min} + m}} )}}}}} & (1)\end{matrix}$

Next, in the brightness distribution extraction process, an attentionblock and blocks around the attention block (hereinafter referred to asa circumference block) are selected (Step S52) based on the averagebrightness value of each block calculated by the above equation (1). Ablock 201 a illustrated by a thick line in FIG. 6A is the attentionblock. This attention block 201 a is selected from blocks 201 having alow average brightness value. Namely, the brightness value of the areawhere dirt is attached is likely to be smaller than the averagebrightness value of the area where dirt is not attached. For thisreason, in the lens-attached matter detection process of the presentembodiment, this tendency is used as a feature of dirt, and theattachment of dirt is determined in the subsequent attached matterdetermination process.

The circumference blocks 201 b located in the outer circumferences ofthe blocks 201 adjacent to the attention block 201 a in the outercircumference of the attention block 201 a are selected as thecircumference blocks 201 b. This is because the dirt often attaches notonly to one block but also to the adjacent blocks, so that it isconsidered that there is no difference in an average brightness valuebetween the attention block 201 a and the blocks adjacent to theattention block. Therefore, the blocks outside the adjacent blocks areselected as the circumference blocks 201 b. In addition, the presentinvention is not limited thereto. When the attachment area of theattached material is small, the blocks 201 adjacent to the attentionblock 201 a can be selected as the circumference blocks 201 b. When theattachment area of the attached material is large, the blocks away fromthe attention block by several blocks can be selected as thecircumference blocks 201 b.

As described above, after selecting the attention block 201 a and thecircumference blocks 201 b, the number of the circumference blocks 201 bhaving an average brightness value higher than that of the attentionblock 201 a is counted (Step S53). In this case, the number ofcircumference blocks is counted by using the gray-scaled image beforebinarization. Next, the ratio of the bright circumference block 201 b(the number of bright circumference blocks/the total number ofcircumference blocks) is calculated (Step S54). As a result, regarding ablock having dirt (attention block 201 a), the ratio of the number ofbright circumference blocks having a high average brightness value isincreased.

Next, the weak edge is counted (Step S55) by using the edge imageextracted by the edge extraction process (step S40). The counting ofthis weak edge is executed by using the image after binarization. Thedirt attached to the lens is likely to be blurred without being focusedand present as a block of weak edges. For this reason, in thelens-attached matter detection process of the present invention, thenumber of weak edges is counted in each block. FIG. 7 illustrates animage of weak edges. In this block, the weak edges are present in aportion surrounded by the inside white line. The number of the countedweak edges is stored in the memory 60.

(Process Time Determination Process)

After the above process is completed to one input image, the processtime determination process (Step S60) in FIG. 16 is executed by thecontroller. When it is determined that a predetermined time has passed,the subroutine proceeds to a next brightness change extraction process(Step S70). When a predetermined time has not passed, the processarea-obtaining process (Step S30), edge extraction process (Step S40)and brightness distribution extraction process (Step S50) are repeated.By repeating Steps S30-S50 multiple times within a predetermined time,information such as the average brightness value, the ratio of brightcircumference blocks and the number of counted weak edges areaccumulated in the memory 60 in chronological order. In this embodiment,the process is executed twenty times at intervals of one second toaccumulate the information. In addition, this time can be freely setaccording to the vehicle information such as a type or vehicle speed orother conditions. For example, during off-road running on a rainy day,attachment of dirt occurs very often. Accordingly, it is possible todetect dirt in a short period of time whereas smooth warning isrequired. For this reason, it is preferable to set a short period oftime. On the other hand, during off-road running on a sunny day,attachment of dirt hardly occurs. Accordingly, it is preferable to set along period of time because it is preferable to accumulate informationfor a long period of time in order to achieve highly accurate detection.

(Brightness Change Extraction Process)

The brightness change extraction process (Step S70) is executed by thebrightness change extractor 38. The details of the brightness changeextraction process will be described with referent to FIG. 8. The dirtattached to the lens hardly moves over time, and the permeability of thedirt is low. For this reason, a variation in the brightness values inthe time direction within the range is reduced. In order to find out achange in pixel values in the time direction, the average brightnessvalue (representative brightness value of block) for a predeterminedtime is accumulated in the memory 60. In other words, the averagebrightness value obtained in the brightness distribution extractionprocess of Step S50 is accumulated in the memory 60 with respect to eachblock. FIG. 8 illustrates an image of record of the average brightnessvalue of each block at regular time intervals.

The representative brightness value E with respect to each block iscalculated by using the following expression (2) based on theaccumulated average brightness value. In the following equation (2),I_(ave) denotes an average brightness value of a block, i denotes ablock number, and N denotes a predetermined time (process time).

$\begin{matrix}\lbrack {{Expression}\mspace{14mu} 2} \rbrack & \; \\{{E(i)} = {\sum\limits_{n = 0}^{N - 1}{{Iave}(i)}}} & (2)\end{matrix}$

Next, the variance V in the time direction is calculated with respect toeach block by using the following expression (3) based on the averagebrightness value with respect to each block calculated by the aboveexpression (2). Namely, the variance in the time direction is calculatedby using the representative brightness value for one cycle previouslyprocessed and accumulated in a predetermined time. The variance V of therepresentative brightness value in the time direction is calculated withrespect to each block. In the following expression (3), I_(ave) denotesan average brightness value of a block, i denotes a block number, and Ndenotes a predetermined time (process time).

$\begin{matrix}\lbrack {{Expression}\mspace{14mu} 3} \rbrack & \; \\{{V(i)} = {\frac{1}{N}{\sum\limits_{n = 0}^{N - 1}( {{{Iave}(i)} - {E(i)}} )^{2}}}} & (3)\end{matrix}$

(Attached Matter Determination Process)

Upon completion of the process for the accumulated information, next,the attached matter determination process (Step S80) is executed by theadhered matter determiner 32. In this embodiment, since dirt isdetected, the attached matter is limited to dirt in this process.However, the attached matter is not limited to dirt in the presentinvention as a dirt determination target, and dirt can be read as eachattached matter, for example, a waterdrop determination target.

The details of the attached matter determination process will bedescribed with reference to the flowchart in FIG. 20. At first, thescore of the dirt determination target is calculated with respect toeach block (Step S81) based on the following information accumulated inthe memory 60.

(a) The number of counted weak edges

(b) Ratio of the number of bright circumference blocks in circumferenceblocks

(c) Variance of an average brightness value with respect to each processblock

More particularly, in the attention block in which the number of countedweak edges is less than a threshold, the score of the dirt determinationtarget is not counted due to a low attachment rate of dirt. When theratio of the number of bright circumference blocks in the circumferenceblocks is higher than the threshold, the adding rate of the score isincreased. Moreover, when the variance of the average brightness valuewith respect to each block is within a predetermined range, the addingrate of the score is increased.

Next, the dirt determination is executed (Step S82) based on thefollowing information. The dirt is determined when any of the followingconditions is satisfied.

(a) Score of dirt determination target is a threshold or more

(b) Variance value with respect to each process block is a threshold orbelow

In the present embodiment, after determining dirt, the outputinformation is generated in the after-described output informationgeneration process, so as to send the dirt information to the cleaner 51or the image-sensing application 52 such as vehicle detection or lanedetection. In the dirt determination, the following information iscalculated as determination results for use in the output informationgeneration process. The calculation result is stored in the memory 60.

(a) Dirt determination result with respect to each process block(attachment of dirt/no attachment of dirt)

(b) Dirt area on process block (unit: block)

Next, the following dirt attachment rates are calculated based on thedirt determination result (Step S83). In addition, the followinginformation is an example, and arbitrary information can be calculatedby the image-sensing application (for example, moving vehicle detectionor parking frame recognition) using dirt attachment information or a wayof dealing with dirt.

(a) Dirt attachment rate in process area in lane detection

(b) Dirt attachment rate in process area in vehicle detection

(c) Dirt attachment rate in process block of automatic cleaningexecution determination area

(Output Information Generation Process)

The output information generation process (Step S90) is executed by theoutput information generator 34. In this case, the output informationfor sending to another application or device is generated based onvarious information calculated in the attached matter determinationprocess. In addition, when the process determined as NO in the vehiclespeed determination process (Step S20) is skipped, the outputinformation (for example, clear information) indicating that thelens-attached matter detection process is performed is output.

For example, the cleaner 51 performs the cleaning process for a lensbased on the output information from the lens-attached matter detector30. The alarm generator 53 and the display 54 draw a driver's attentionby generating an alarm and displaying a warning lamp and a warningcharacter. Moreover, in the image-sensing application 52 such as vehicledetection, traffic lane detection or the like, the output information isused for the determination information of each process. A process to beexecuted by sending information to any of these devices or applicationsis arbitrary and appropriately selected depending on the configurationsof the vehicle system 10.

As described above, in the first embodiment, the presence or absence ofthe attachment of the dirt can be detected with high accuracy.Consequently, for example, dirt on a lens can be effectively informed toa driver, automatic cleaner or the like so as to smoothly eliminatelens-attached matter, and an image-sensing application such as vehicledetection using a camera can be preferably performed so as to enablehigh-performance vehicle running. Since attached matter is detected byboth of a brightness value and variation with time, an imaging objectexcept lens-attached matter such as a landscape, guide rail, parapet, orsidewalk is eliminated from the attention area, so that highly accuratedetection can be achieved. As a result, it becomes unnecessary to informa driver about dirt in an undesirable manner or to perform automaticcleaning, so that effective preferable vehicle running can beeffectively performed.

In addition, in the first embodiment, in the edge extraction process orthe brightness distribution process, each extraction data is stored andaccumulated in the memory 60, and a statistic process is executed in thebrightness change extraction process after the elapse of a certainperiod of time. However, the present invention is not limited thereto,the brightness change extraction process can store the data extracted bythe edge extraction process or the brightness distribution process.Moreover, the brightness change extraction process can monitor a timewith a timer, and execute the statistic process after the elapse of acertain period of time.

Second Embodiment

A vehicle system including a lens-attached matter detector according tothe second embodiment will be described. The configurations of thevehicle system in the second embodiment are the same as those of thevehicle system in the first embodiment except that the vehicle system inthe second embodiment includes a mask processor which controls falsedetection. Therefore, the configurations of the vehicle system of thesecond embodiment will be described with reference to FIG. 2, and thesame reference numbers are applied to the configurations which are thesame as those in the first embodiment; thus, the detailed descriptionthereof will be omitted.

As illustrated in FIG. 2, the vehicle system 10 including alens-attached matter detector of the second embodiment includes thecamera (imaging device) 20, lens-attached matter detector 30, cleaner51, image-sensing application 52, alarm generator 53, display 54, andmemory 60.

The image processor 31 of the second embodiment includes the processarea obtainer 35, edge extractor 36, brightness distribution extractor37, brightness change detector 38, and a mask processor 39 asillustrated by the solid line and the dotted line in FIG. 2.

The lens-attached matter detection process in the lens-attached matterdetector 30 according to the second embodiment will be described withreference to the drawings. In this embodiment, dirt is detected aslens-attached matter. The same description and the same flowcharts areused for the processes similar to those in the first embodiment.

As illustrated in the flowchart in FIG. 21, the lens-attached matterdetection process of the second embodiment includes a various vehicleinformation-obtaining process (Step S110), process prosecutiondetermination process (Step S120), process area-obtaining process (StepS130), edge extraction process (Step S140), mask process (Step S150),brightness distribution extraction process (Step S160), process timedetermination process (Step S170), brightness change extraction process(Step S180), attached matter determination process (Step S190), andoutput information generation process (Step S200). These processes arecontrolled by the controller (CPU).

(Various Vehicle Information-Obtaining Process)

In the various vehicle information-obtaining process (Step S110), atfirst, various vehicle information such as vehicle speed information,on-road or off-road information, or day-and-night information isobtained. Weather information such as sunny, rain or snow can beobtained. The on-road or off-road information can be obtained by anydevice. It can be determined based on the image of the camera 20 or theGPS information, for example, or can be determined based on carnavigation information. The day-and-night information is determinedbased on the gain adjustment value by the gain adjuster 23 relative tothe image obtained by the imaging element 22 of the camera 20. Theincrease in the gain adjustment value relative to the image indicatesthat a dark image is obtained, which can be determined as night-time. Onthe other hand, the decrease in the gain adjustment value indicates thata bright image is obtained, which can be determined as day-time. Thevehicle speed information is obtained from the signal information fromthe vehicle speed sensor. The weather information can be determinedbased on rainfall by using a driving signal of a windshield wiper or canbe obtained from car navigation information including a communicationfunction, or the like.

(Process Prosecution Determination Process)

In the process prosecution determination process (Step S120), it isdetermined whether or not the lens-attached matter detection process isprosecuted based on the obtained various vehicle information. In theday-and-night determination, when it is determined as a daytime based onthe gain adjustment, the cleaning determination is performed. When it isdetermined as night-time or the like in addition to day-time, theprocesses in Steps S130-S190 are skipped, and the subroutine proceeds tothe output information generation process (Step S200), and then, thewhole processes are completed.

In the second embodiment, the lens-attached matter detection process isactivated with the engine startup or the running start of the vehicle 1as a trigger, and the prosecution, completion, and interruption of theabove-described lens-attached matter detection process are determinedbased on day and night and the cleaning condition. However, the presentinvention is not limited thereto. Any condition can be a determinationtarget according to the running condition or the like. For example, asdescribed in the first embodiment, the determination can be made basedon the vehicle speed. The determination can be made based on an on roador an off road, and the present process can be activated in an off road,for example.

(Process Area-Obtaining Process)

The process area-obtaining process (Step S130) is performed by theprocess area obtainer 35, and the process target area is set from theinput image from the camera 20 so as to be blocked. In this embodiment,an area including the process area in the vehicle detection, the processarea in the lane detection and the automatic cleaning determination areais set as a process target area. Since the outline of the processarea-obtaining process is similar to that of the first embodiment, thedetailed description thereof will be omitted.

(Edge Extraction Process)

The edge extraction process (Step S140) is performed by the edgeextractor 36 and the mask processor 39. The edge extraction process inthe present embodiment is a process similar to that in the firstembodiment except that the edge extractor 36 extracts an edge in theedge extraction (step S42) by using an image in which a lane marker(hereinafter, referred to as a lane) is masked by the mask processor 39.Therefore, the detailed description for the same processes is omitted,and the mask process will be only described in this embodiment. An imageof a lane indicated on a road is sharply shot by the camera 20, so thata weak edge is hardly detected in the border with the road. However,when the lane is unclear or tainted, the edge portion of the lane may beblurred, and the weak edge may be sometimes detected, so that it becomesdifficult to discriminate the weak edge of the lane from the weak edgeof the lens-attached matter. For this reason, the mask process for thelane is executed for improving the detection accuracy of thelens-attached matter.

The mask process for the lane will be described with reference to FIGS.9A-10. FIG. 9C illustrates the image of the lane on the shot image (realview) of the back of the vehicle. In order to control the edgeextraction of symmetrical lanes L, R generating from a disappearancepoint, the image is divided into half right and left, and a filter forthe left half image and a filter for the right half image are applied. Aleft area edge filter (edge filter whose brightness is increased fromthe upper right to the lower left) illustrated in FIG. 9A is a filterfor avoiding the extraction of the edge of the lane in the L-portion inFIG. 9C. A right area edge filter (edge filter whose brightness isincreased from the upper left to the lower right) illustrated in FIG. 9Bis a filter for avoiding the extraction of the edge of the lane in theR-portion in FIG. 9C. The edge filters are for decreasing noise edges byincreasing a size of a filter.

As illustrated in FIG. 10, after controlling the lane by applying theright area filter and the left area filter to the input image dividedright and left, the edge of the right area and the edge of the left areaare extracted. By combining the edge images independently obtained inthe right area and the left area, the edge image is generated. Inaddition, the mask for the lane can be performed when executing the maskprocess for the after-described light source or the like. However, bymasking the lane with a process which extracts the edge image, the edgeextraction process can be improved and the speed of the edge extractionprocess can be improved, and these processes can be improved because itbecomes unnecessary to count an unnecessary edge in the subsequentprocesses.

Next, the threshold process and the binarization process are executedbased on the weak edge image generated above to execute the noiseelimination (refer to Steps S43, S44 in FIG. 18). These processes aresimilar to those in the first embodiment; thus, the detailed descriptionthereof will be omitted.

(Mask Process)

The mask process (Step S150) is performed by the mask processor 39. Inthis embodiment, the mask process for a track and the mask process for alight source are executed. The details of the mask process will bedescribed with reference to FIGS. 11-12B.

The various processes for producing a track mask image illustrated inFIG. 11 will be described. Tracks generated on a road surface include asnow track or a rain track. In an off-road, a track due to a tire trackon a road surface is generated. The contours of these tracks aresometimes extracted as weak edges, and it may be difficult todistinguish the weak edges of the counters of the tracks from the weakedge of the dirt. For this reason, the weak edge extraction may beaffected (noise), so that the area including the tracks is masked inorder to effectively and highly accurately extract the weak edge. Inaddition, the tracks are likely to be generated during off-road runningon a snow day or a rain day. Therefore, during on-road running on aclear day which hardly generates tracks, the track image process and thetrack mask process can be skipped. Depending on the difference of theseconditions, the contrasting density of a track is changed, so that theparameter for use in the generation process of the mask image can beadjusted in accordance with the difference in the conditions.

In the track detection, the contour position of the track is specifiedin the input image illustrated in (a) of FIG. 11. In order to specifythe contour position of the track, the input images for a predeterminedtime are compared, and the area where the weak edges are temporarily andspatially continued in the vehicle traveling direction is detected. Theprocess target area is divided into a plurality of small frames, and thepixels which detect the edges two frames in a row are displayed by whiteas illustrated in (b) of FIG. 11. A line in which the edges arecontinued in the Z axis direction on the world coordinate (in this case,the X axis is the vehicle width direction, the Y axis is the verticaldirection, and the Z axis is the vehicle traveling direction) isextracted, and this line is set as the mask target area. The mask imageobtained by this process is illustrated in (c) of FIG. 11. The detailsof these processes will be described below.

At first, the process target area is divided into a plurality of smallframes as illustrated in (b) of FIG. 11, and the presence or absence ofthe edge in the frame is confirmed, and counted. Moreover, theprojection in which each frame is arranged on the Z axis and the X axison the world coordinate is obtained. In this case, the presence orabsence of the edge (0/1) not the number of edges is projected.

Next, the presence or absence of the track is determined with respect toeach line based on the above tract detection data. The trackdetermination is performed according to the following procedures byusing the number of areas having an edge projected on the X axis.

(A) Track Determination

In this determination, the determination differs between the outer edgeportion of the process target area and the portion except the outer edgeportion. In addition, the outer edge portion is defined as apredetermined area of the left end side and a predetermined area of theright end side in the X-axis projection. In the portion except the outeredge portion, it is determined that the line includes the track whensatisfying the following conditions (a), (b). In the outer edge portion,it is determined that the line includes the track when satisfying thefollowing condition (c).

(a) The number of areas having an edge in a line is threshold 1 or more.

(b) Two lines each on the right and left sides in a line include a linein which the number of areas including an edge in a line is less thanthreshold 2.

(c) The number of areas including an edge in a line is threshold 3 ormore.

(B) Track Continuation Determination

In the determination of the above (A), in a case that no track isdetermined (in a case that the conditions of the above (A) are notsatisfied), and a track is recently detected, the track determination ofthe line is continued when satisfying both of the following conditions(d), (e). A period for continuing a track is twice (one second), and thecontinuation period is reset when the track determination is continuedby this process.

(d) The number of areas having an edge in a line is ½ or more ofthreshold 1.

(e) Two lines each on the right and left sides in lines include a linein which the number of areas having an edge is less than ½ of threshold2.

An image which masks a line determined as including a track is producedbased on the track detection result. The mask image is produced inaccordance with the following procedures. The track mask imageillustrated in (c) of FIG. 11 is produced in accordance with thefollowing procedures.

(a) The output image is filled with white.

(b) When there is a line determined as the presence of a track, thatarea is blacked out.

Next, the various processes for masking an area including a light sourcewill be described. When a light source such as sun exits at the back ofthe own vehicle, the weak edge of the light source is detected in theboarder of the road surface reflection area. This weak edge due to thelight source is hardly distinguished from the weak edge of the dirt, andmay affect the performance of the lens-attached matter detectionprocess. The mask process for the light source is executed forcontrolling such a negative influence. FIG. 12A illustrates the image ofthe light source and the road surface reflection. FIG. 12B illustratesthe light source mask image in which the light source is masked. Aprocedure for producing the light source mask image relative to such aninput image is described as follows.

The following information is used in this embodiment as information of alight source area.

(a) West sun determination result

(b) Diffusion and reflection area

Since the information of the above (a), (b) is an area including apredetermined brightness value or more, the light source area can beobtained by binarizing the input image with a predetermined brightnessvalue. In addition, the information of the light source area is notlimited thereto, and other information can be used. As described above,in the process block which overlaps with the light source area, thescore of the dirt determination target is cleared. Namely, whenexecuting the after-described attached matter determination process, theprocess block which overlaps with the light source area is set aside forcounting, so that the area (light source area) except the dirt isprevented from being counted.

As described above, in the present embodiment, the mask process isexecuted for the area including a track and the area including a lightsource. However, the present invention is not limited thereto, and anytarget which can be noise can be masked for effectively and highlyaccurately extracting an edge. Another example of a mask target includesown vehicle shadow illustrated in FIGS. 12A, 12B and a vehicle bodyillustrated in FIG. 13. The own vehicle shadow is masked because theweak edge is detected by the contour of the own vehicle shadow, and suchdetection affects the detection of the weak edge of the dirt. For thisreason, in this process, as illustrated in FIG. 13, a mask image formasking the contour of the own vehicle shadow is created. Moreover, theown vehicle shadow generated in the input image is not fixed, and theprojected image generated on the image differs according to a time or adirection. In this case, the mask image is created by accumulating theinput images for a predetermined time, and specifying the own vehicleshadow from the portion without having a change over time. The ownvehicle shadow can be specified based on the shape of the projectedimage or the previously prepared sample image of the own vehicle shadow.There may be a case in which the contour of the shadow changes inaccordance with intensity of light depending on weather, or the shadowis not obtained on a cloudy day or the like. For this reason, theprocess of creating the mask image of the own vehicle shadow can bechanged or can be skipped based on the weather information.

In contrast to this, since the image of the vehicle body is fixed basedon the attachment position of the camera 20 or the like, it becomesunnecessary to execute a time-dependent process, and the mask imageillustrated in FIG. 14 may be prepared in advance. The image of thevehicle body can be obtained from the input image as an initial processupon the activation of the lens-attached matter detection process, andthe mask image can be created. As described above, the mask process fora lane can be executed in this step. FIG. 15 illustrates the weak edgeimage before the mask process and the weak edge image after the maskprocess. FIG. 15 illustrates the images after the mask process bycreating the track-masked image, own vehicle shadow-masked image andvehicle body-masked image.

(Brightness Distribution Extraction Process)

The brightness distribution extraction process (Step S160) is executedby the brightness distribution extractor 37. The brightness distributionextraction process in the second embodiment is similar to that in thefirst embodiment except that the process is not executed for the maskedblock. Consequently, the detailed description of the similar processesis omitted, and the process different from that in the first embodimentwill be described below.

In the first embodiment, when selecting the attention block and thecircumference blocks (Step S52), all of the blocks located in the outercircumference of the blocks adjacent to the attention blocks areselected as the circumference blocks for process targets. However, inthe second embodiment, as illustrated in FIG. 6B, the circumferenceblocks 201 b are selected from the process target blocks 201 c. Thus,the dotted line portions are not selected as the circumference blocks201 b. These process target blocks 201 c are portions which are notmasked by the mask process (Step S150). Namely, the masked portions arenot set as process target blocks. Owing to such a process, a dark blockexcept dirt is prevented from being counted, while the number of processtarget blocks is reduced. Accordingly, the process speed can beimproved. In the process which counts a weak edge (Step S55), since theweak edge of the masked portion is not counted, the process speed can bealso improved.

(Process Time Determination Process)

In the process time determination process (Step S170) according to thesecond embodiment, the time interval for executing each process and theaccumulation of information is used depending on an on road or an offroad in the determination of the elapse of a predetermined time. Namely,in an off road, the attachment of the dirt easily occurs, so that thepresence or absence of the dirt can be detected by the informationaccumulated for a short period of time. On the other hand, in an onroad, the attachment of the dirt hardly occurs compared to an off road,so that a long time interval is set, and the determination is executedby accumulating the information for a long period of time. In thisembodiment, the time interval for accumulating the information is setdepending on an on road or an off road because the presence or absenceof the dirt is detected. However, the present invention is not limitedthereto. In a case of detecting waterdrops, for example, the timeinterval can be adjusted depending on weather. The time interval fordetecting the presence or absence of the dirt can be adjusted dependingon not only a road condition but also weather.

(Brightness Change Extraction Process)

The brightness change extraction process (Step S180) is executed by thebrightness change extractor 38. In the brightness change extractionprocess in the second embodiment, the process similar to that in thefirst embodiment is executed. Thus, the detailed description thereofwill be omitted. This process is not executed for the masked block, sothat the process speed can be improved.

After the accumulated information is sufficiently obtained, the attachedmatter determination process is executed by the attached matterdeterminer 32 (Step S190). The attached matter determination process inthe second embodiment is similar to the process in the first embodimentexcept that the masked block is not counted. Thus, the description forthe similar processes is omitted, and the process different from that inthe first embodiment will be described below.

In the calculation of the score of the dirt determination target withrespect to each block (Step S81), the block masked by theabove-described each mask process is not set as a count target for eachprocess. Namely, as described above, the process block which overlapswith the light source area, the process block which overlaps with thetrack, the process block which overlaps with the own vehicle shadow andthe process block which overlaps with the own vehicle body are set asidefor counting a dirt determination target.

(Output Information Generation Process)

The output information generation process (Step S200) is executed by theoutput information generator 34. In the output information generationprocess in the second embodiment, the process similar to that in thefirst embodiment is executed. Thus, the detailed description thereofwill be omitted.

As described above, in the second embodiment, even when a road surfacecondition (for example, lane marker, track, own vehicle shadow) orillumination environment (for example, light source such as sun light orvehicle light), which is likely to be false-recognized as dirt, occurs,the mask process which eliminates an imaging target except lens-attachedmatter from the attention block is executed, so that the presence orabsence of the attachment of the dirt can be detected with a highaccuracy. By eliminating the mask area from the dirt determinationprocess target, the whole lens-attached matter detection process can beeffectively executed at a high speed.

In the first and second embodiments, an example which detects dirt isdescribed. However, the present invention is not limited to thedetection of the dirt. The present invention can be used for a detectoror a detection process for another lens-attached matter such aswaterdrops, dust, paint or bird dropping. For example, when waterdropssuch as rain or fog attach to a lens, a problem, for example, a strainimage or miss-focusing occurs, so that the operation of theimage-sensing application such as vehicle detection or lane detectionmay be affected. For this reason, it becomes necessary to draw driver'sattention by detecting the attachment of the waterdrops to blow out theattachment by air, for example.

In each embodiment, since the dirt is detected, the presence of theattachment of the dirt is determined when the average brightness valueof the attention block is lower than that of the circumference block inthe brightness distribution extraction process and the change in theaverage brightness value is small in the brightness change extractionprocess. More specifically, the brightness is decreased because the dirthas a low light permeability and is brackish. On the other hand, thewaterdrops have a high light permeability and brightness which issubstantially similar to that in the circumference, but the brightnessof the waterdrops is likely to be changed over time. Moreover, the weakedge is extracted in the outer circumference of the waterdrops when thewaterdrops attach to a lens. However, the degree of the edge intensitymay differ from that of the dirt. Therefore, the attachment of thewaterdrops can be determined when the average brightness value of theattention block is equal to that of the circumference block or a changein the average brightness value is large in the brightness changeextraction process in a case that the weak edge is detected.

Consequently, in the detection of the attachment of waterdrops, it ispreferable to use a threshold suitable for the waterdrops whenextracting the weak edge by the edge extraction process in the first andsecond embodiments. When comparing the average brightness value of theattention block with that of the circumference block by the brightnesschange extraction process, the determination is based on whether or notthe brightness is the same or whether or not the change in thebrightness by the brightness change extraction process is large. Asdescribed above, it is preferable to configure the lens-attached matterdetection process with the logic which executes a process according to atype of attached matter. Moreover, it is preferable to configure thevehicle system which executes the most suitable process according to atype of attached matter.

According to the above-described lens-attached matter detector, onlylens-attached matter such as dirt, dust, or waterdrops, which isattached to a lens of a camera, for example, can be detected with a highaccuracy. Therefore, for example, a stain on a lens can be smoothlyinformed to a driver or an automatic cleaning system, so thatlens-attached matter can be smoothly eliminated. Moreover, the imagingsensing application such as vehicle detection using a camera can bepreferably performed. Furthermore, only lens-attached matter can bedetected with a high accuracy without detecting attached matter exceptlens-attached matter because the lens-attached matter is detected basedon the comparison between the brightness value of the attention area andthe brightness value of the circumference area. Consequently, theprocess efficiency can be improved without unnecessarily informing dirtto a driver or unnecessarily performing automatic cleaning.

As described above, the lens-attached matter detector may include themask processor which eliminates from the attention area an area exceptlens-attached matter in the area whose edge intensity is within athreshold range extracted by the edge extractor. With thisconfiguration, when a road surface condition (for example, lane marker,track, or own vehicle shadow) or an illumination environment (lightsource such as sun light or vehicle light), which is likely to befalse-detected as dirt, in which a time series variation in a brightnessvalue is small, and the brightness value of the attention area issmaller than the brightness value of the circumference area in the areawhose edge intensity is within the threshold range occurs, an imagingobject except lens-attached matter can be eliminated from the attentionarea. Therefore, the determination of the presence or absence of thelens-attached matter by the lens-attached matter determiner can beexecuted with a high accuracy.

Moreover, according to the embodiments of the present invention, thelens-attached matter detector and the lens-attached matter detectionmethod capable of detecting only lens-attached matter such as dirt,dust, or waterdrops which is attached to a lens of a camera, forexample, and the vehicle system including such a lens-attached matterdetector can be provided.

Although the embodiments of the present invention have been describedabove, the present invention is not limited thereto. It should beappreciated that variations may be made in the embodiments described bypersons skilled in the art without departing from the scope of thepresent invention.

What is claimed is:
 1. A lens-attached matter detector, comprising: anedge extractor configured to create an edge image based on an inputimage from an imager having a lens, divide the edge image into aplurality of areas including a plurality of pixels, and extract an areawhose edge intensity is a threshold range as an attention area; abrightness distribution extractor configured to obtain a brightnessvalue of the attention area and a brightness value of a circumferencearea; a brightness change extractor configured to obtain the brightnessvalue of the attention area and the brightness value of thecircumference area obtained by the brightness distribution extractor fora predetermined time interval, and obtain a time series variation in thebrightness value of the attention area based on the brightness value ofthe attention area for the predetermined time interval; and an attachedmatter determiner configured to determine the presence or absence ofattached matter based on the time series variation in the brightnessvalue of the attention area.
 2. The lens-attached matter detectoraccording to claim 1, further comprising a mask processor configured toeliminate from the attention area the area except lens-attached matterin the area whose edge intensity is the threshold range extracted by theedge extractor.
 3. The lens-attached matter detector according to claim2, wherein the mask processor is configured to eliminate from theattention area an area including an edge extending in a direction towarda disappearance point of the input image.
 4. The lens-attached matterdetector according to claim 2, wherein the mask processor is configuredto eliminate from the attention area an area including a track on a roadsurface, which is generated on the input image.
 5. The lens-attachedmatter detector according to claim 2, wherein the mask processor isconfigured to eliminate from the attention area an area including ashadow of an attached object provided with the imager, which isgenerated on the input image.
 6. The lens-attached matter detectoraccording to claim 1, wherein the mask processor is configured toeliminate from the attention area an area including an image of a lightsource, which is generated on the input image.
 7. The lens-attachedmatter detector according to claim 1, wherein when the brightness valueof the attention area obtained by the brightness distribution extractoris smaller than a predetermined value by being compared with thecircumferential area, and the time series variation in the brightnessvalue obtained by the brightness change extractor is smaller than apredetermined value, the attached matter determiner is configured todetermine that attached matter having a low permeability is attached tothe attention area.
 8. The lens-attached matter detector according toclaim 1, wherein when the brightness value of the attention areaobtained by the brightness distribution extractor is larger than apredetermined value by being compared with the circumferential area, andthe time series variation in the brightness value obtained by thebrightness change extractor is larger than a predetermined value, theattached matter determiner is configured to determine that attachedmatter having a high permeability is attached to the attention area. 9.A lens-attached matter detection method which is executed by thelens-attached matter detector according to claim 1, comprising an edgeextraction process of creating an edge image based on an input image,dividing the edge image into a plurality of areas including a pluralityof pixels, and extracting an area whose edge intensity is a thresholdrange as an attention area; a brightness value distribution extractionprocess of obtaining a brightness value of the attention area and abrightness value of a circumference area of the attention area; abrightness value extraction process of obtaining the brightness value ofthe attention area and the brightness value of the circumference areaobtained by the brightness value distribution extraction process for apredetermined time interval, and obtaining a time series variation inthe brightness value of the attention area based on the brightness valueof the attention area for the predetermined time interval; and anattached matter determination process of determining the presence orabsence of attached matter based on the time series variation in thebrightness value of the attention area.
 10. The lens-attached matterdetection method according to claim 9, further comprising a vehicleinformation-obtaining process of obtaining vehicle information includingat least vehicle speed information, wherein all of the processes arecompleted when the vehicle information-obtaining process determines thata vehicle speed is a threshold or below based on the vehicle speedinformation.
 11. The lens-attached matter detection method according toclaim 9, wherein all of the processes are completed when a gainadjustment value of the input image is a threshold or more.
 12. Avehicle system, comprising a lens-attached matter detector according toclaim 1; an imager provided in a vehicle, having a lens, and configuredto image a circumference of the vehicle; and at least one applicationconfigured to operate based on detection information of thelens-attached matter detected by the lens-attached matter detectorrelative to the input image shot by the imager.
 13. The vehicle systemaccording to claim 12, wherein the lens-attached matter detector isconfigured to detect the lens-attached matter relative to only an areafor use in the application in the input image shot by the lens.